Specification

Quality KIT

Quality Data Offers at EDC

When a Data Consumer calls the Catalog of a Data Provider, the Data Provider must signal in each Data Offer what exactly a Consumer could negotiate for. Data Offers in the Catalog are sorted by dcat:Datasets which are registered in the EDC- Management API as edc:Asset. Each Asset has private and public properties. The public properties are shown in the catalog and give hints to the Data Consumer what API and data they may expect. There are some properties that are mandatory for the entire Catena-X network and some that are mandatory only in specific Business Scenarios (like Quality).

The dataAddress object's structure is determined by the dataplane implementation as it configures the details of the data transfer. They are not visible via the catalog.

The following suggestion is a non-standardized draft how Assets (and thus by proxy, dcat:Datasets) should be registered in the Quality Use-Case.

{
  "@context": {
    "cx-taxo": "https://w3id.org/catenax/taxonomy#",
    "cx-common": "https://w3id.org/catenax/ontology/common#",
    "dct": "https://purl.org/dc/terms/",
    "dcat": "http://www.w3.org/ns/dcat#",
    "edc": "https://w3id.org/edc/v0.0.1/ns/"
  },
  "@id": "someId",
  "@type": "edc:Asset",
  "edc:properties": {
    "dct:type": {
      "@id": "cx-taxo:QualityAsset"
    },
    "cx-common:version": "1.0",
    "dct:language": {
      "@id": "https://w3id.org/idsa/code/EN"
    },
    "dcat:qualifiedRelation": {
      "dct:isPartOf": {
        "@id": "f7574ad6-95ee-46e2-8a45-6fa1782ba426"
      }
    },
    "dct:conformsTo": {
      "@id": "urn:samm:io.catenax.vehicle.product_description:3.0.0#ProductDescription"
    },
    "dct:description": "TBD",
    "dct:format": "application/octet-stream;type=parquet-snappy",
    "edc:type": "AmazonS3"
  },
  "edc:dataAddress": {
    "@type": "edc:DataAddress",
    "edc:type": "AmazonS3",
    "edc:region": "eu-west-1",
    "edc:bucketName": "int-xcod-quality-aspect-models-eu-west-1",
    "edc:keyName": "myCompany/myTag/QualityTask.parquet",
    "edc:accessKeyId": "…",
    "edc:secretAccessKey": "…"
  }
}

S3 Data Address

This section is not use-case specific but since the EDC's AmazonS3 dataplane is basically undocumented, here is an explanation:

Property	Value	Description
edc:type	"AmazonS3"	This shows which data source the Data Plane will query. It also determines what other content the dataAddress object must hold.
edc:region	"eu-west-1"	This property represents the AWS-region where the source bucket is located.
edc:bucketName	"provider-quality-bucket"	This is the name of the source bucket that the data to-be-transferred resides in.
edc:keyName	"path/through/provider/s3"	This is the path of the file that shall be offered to the dataspace.
edc:accessKeyId	"<keyId>"	Amazon S3 uses this property similarly to how oauth2 client credentials use the clientId. Note that this can also be set during deployment-time for the whole S3-dataplane. If it's set here, it will override the default config.
edc:secretAccessKey	"<secretAccessKey>"	This secret is used similarly to a clientSecret in oauth2 client credentials.

Properties

Property	Value	Optional	Description
https://purl.org/dc/terms/type	{"@id": "cx-taxo:QualityAsset"}		CX-0018 mandates the usage of the dct:type property to signal what kind of Asset a consumer can expect behind a dcat:Dataset. In the Quality Use-Case, this is identified as https://w3id.org/catenax/taxonomy#QualityAsset. The expected payload this API serves is determined by the dcat:conformsTo property.
https://purl.org/dc/terms/language	{"@id": "https://w3id.org/idsa/code/EN"}	x	This property is QM-specific. As it points to an IRI, it must be embedded in a json-object with the @id key. The use of this is unclear.
https://purl.org/dc/terms/format	"application/octet-stream;type=parquet-snappy"		This property is QM-specific. dct:format usually points to the correct IANA Media Type. As currently only parquet files are used, the type application/octet-stream with the added property type=parquet-snappy must be used. The syntax is expained here. If in the future csv shall be supported, the value could also be text/csv.
https://purl.org/dc/terms/description	<whatever>	x	This property is QM-specific. For human-readable content, rdfs:comment is the usual property but would introduce another namespace so the dct-native property is chosen here.
https://purl.org/dc/terms/conformsTo	{"@id":"<urnOfTheCorrespondingAspectModel>"}		This property is QM-specific. It holds the exact aspect-model-URN that defines the schema of the presented dataset including its version. The version in here refers to the data model's version while the EDC-property cx-common:version defines the version of the underlying API serving the data.
http://www.w3.org/ns/dcat#qualifiedRelation	{"dct:isPartOf": {"@id": "<idOfTheCorrespondingQualityTask>"}}		This property is QM-specific. All Asset types defined in this Kit must include this property as it links the data behind an asset with the correct QualityTask. Note that the id of the QualityTask must be used, not the id of the EDC-Asset shielding said QualityTask.
https://w3id.org/edc/v0.0.1/ns/type	AmazonS3		This property signifies the EDC dataplane that the QM data will be transferred over. The expectation that this would be signaled via the dcat:DataSet-dcat:distribution property of the catalog currently isn't implemented in the EDC. Thus the data must be replicated here and is presented via the same property that the consumer-side transferprocesses API uses for this same signal.
https://w3id.org/catenax/ontology/common#version	"1.0"		CX-0018 recommends to use cx-common:version to signal the API's version. Here, the API's version is equivalent to the version of the CX-standard for the Quality domain. Creation is currently in progress as CX-0123 v1.0.0. In this EDC-property, only major and minor increments should be added.

For the process described in the KIT Quality, data exchange between the participating partner companies is necessarily to be done for large vehicle and product populations. The data exchange therefore should be done as a file download via EDC according to the following specifications.

Asset File type

It is recommended to create and transfer the files in the type parquet. Only attachments for the Quality task (Quality Task Attachment) should be created in ZIP format.

Asset Transfers via EDC

File transfer is recommended to be done via EDC S3 plane, The transfer via EDC http data plane is not recommended due to the big data size.

Asset consumption

File flattening rules: The data provided in the asset is build from 4 to 6 structures. To assure a secure and smooth exchange flatting rules for the file (csv / xls / Parquet / json) must be applied. This includes checks for format and possible values for each column and will be part of next version regulations. If the rules are not applied correctly the mapping of content will not be possible without manual handling effort.

Sample Data

Standard version from: 09.2023

In the following, example data for the standardized data models are provided as download in zip format. The sample data is generated according the current standards. It contains a virtual fleet of 50.000 vehicles where two quality issues are implemented.

• Production failure of product "zehn" at Tier 1
• Specification failure

Quality Task Sample Data

Github Link to semantic data model: CX-00036 Quality Task

Filedownload: 20230704-io.catenax.quality_task.zip

As data provider please add the JSON file from folder tesdata_CX32 as EDC asset id to EDC REST data plane: CX_release32_qualitytask_100_testdata_100_json

As data provider please add the PARQUET file from folder tesdata_CX32 as EDC asset id to EDC S3 data plane: CX_release32_qualitytask_100_testdata_100_parquet

---

Quality Task Attachment Sample Data

Github Link to semantic data model: CX-00092 Quality Task Attachment

Filedownload: 20230718-io.catenax.quality_task_attachment.zip

As data provider please add the PARQUET file from folder tesdata_CX32 as EDC asset id to EDC S3 data plane: CX_release32_qualitytaskattachment_100_testdata_100_zip

---

Fleet Vehicles Sample Data

Github Link to semantic data models:

CX-00037 Vehicle Product Description

CX-00091 Fleet Vehicles

Filedownload: 20230707-io.catenax.fleet.vehicles.zip

As data provider please add the JSON file from folder tesdata_CX32 as EDC asset id to EDC REST data plane: CX_release32_fleetvehicles_100_testdata_100_json

As data provider please add the PARQUET file from folder tesdata_CX32 as EDC asset id to EDC S3 data plane: CX_release32_fleetvehicles_100_testdata_100_parquet

---

Fleet Diagnostic Sample Data

Github Link to semantic data model: CX-00038 Fleet Diagnostic Data

Filedownload: 20230707-io.catenax.fleet.diagnostic_data.zip

As data provider please add the JSON file from folder tesdata_CX32 as EDC asset id to EDC REST data plane: CX_release32_fleetdiagnostic_100_testdata_100_json

As data provider please add the PARQUET file from folder tesdata_CX32 as EDC asset id to EDC S3 data plane: CX_release32_fleetdiagnostic_100_testdata_100_parquet

---

Fleet Claim Sample Data

Github Link to semantic data model: CX-00039 Fleet Claim Data

Filedownload: 20230707-io.catenax.fleet.claim_data.zip

As data provider please add the JSON file from folder tesdata_CX32 as EDC asset id to EDC REST data plane: CX_release32_fleetclaim_100_testdata_100_json

As data provider please add the PARQUET file from folder tesdata_CX32 as EDC asset id to EDC S3 data plane: CX_release32_fleetclaim_100_testdata_100_parquet

---

Manufactured Parts Quality Information Sample Data

Github Link to semantic data model: CX-00041 Manufactured Parts Quality Information

Filedownload: 20230707-io.catenax.manufactured_parts_quality_information.zip

As data provider please add the JSON file from folder tesdata_CX32 as EDC asset id to EDC REST data plane: CX_release32_manufacturingqualityinformation_100_testdata_100_json

As data provider please add the PARQUET file from folder tesdata_CX32 as EDC asset id to EDC S3 data plane: CX_release32_manufacturingqualityinformation_100_testdata_100_parquet

---

Parts Analyses Sample Data

Github Link to semantic data model: CX-00040 Parts Analyses

Filedownload: 20230704-io.catenax.parts_analyses.zip

As data provider please add the JSON file from folder tesdata_CX32 as EDC asset id to EDC REST data plane: CX_release32_partsanalyses_200_testdata_100_json

As data provider please add the PARQUET file from folder tesdata_CX32 as EDC asset id to EDC S3 data plane: CX_release32_partsanalyses_200_testdata_100_parquet